To anyone filing bug reports: Please leave a copy at https://openradar.appspot.com/ so that others can see what the problem is you reported.

The issue was a symlink in my app: The Info.plist points to a file for the app icon. That file is a relative symlink to an existing icon file. This used to work for many years, and the Finder has no problem with it, either. Just Apple's recent verification process stumbles over it, without sending an error msg about it to me.

Specifically comparing searchfs vs getatrrlistbulk, if you end up having read the entire catalog structure off disk, then that cost can mask any performance difference. I just did a new comparison: I compared how searchfs compares vs. recursive dir scan on HFS+ vs. APFS. On SSDs, of course. Always searching for a non-existing file name (so that no extra attributes need to be collected). A search on a HFS+ vol with 8.8M items (6.6M files, 2.2M dirs) on a fast Intel system: searchfs: 40s find: 270s And on a HFS+ vol with 2.6M items (2M files, 600k dirs), Mac Mini 2018: searchfs: 11s find: 70s We can see that the search locally in the FS makes a huge different here. About factor 6 each time. Probably due to the many kernel-user space transitions. Now a search on APFS, with 1.75M items (1.5M files and 250k dirs), on an M4 Mini: searchfs: 13s find: 35s And APFS with 490k items (360k files, 130k dirs), on an Intel Mini 2018: searchfs: 9s find: 40s So, there's still a difference, but smaller. Which may suggest that there's still room for improvement on the searchfs part, or not. But it seems that on APFS everything (both searchfs and readdir etc) is significantly slower compared to HFS+, which is probably due to the dual-tree nature of APFS that requires more reads from the medium. With this in mind, the difference of searchfs performance of HFS+ vs. APFS becomes smaller, and so my bug report (radar 35052030, FB5367373) isn't really an issue. I've justed close it. I suggest we put this to rest. There's nothing to be done about this anyway. I'd rather have someone focus on how to make the Finder be faster when browsing the /Applications folder, e.g. by avoiding needless repeat calls into the FS, because that affects everyone, and to a point where one can get impatient (just try to mount another Mac's startup volume over SMB and browse that Mac's Apps folder over the network and it's time to make a coffee).

[quote='830819022, DTS Engineer, /thread/776198?answerId=830819022#830819022'] However, the real benefit here isn't local filesystem, it's remote file system case where the core cost isn't the direct syscall cost, but the cost of transferring all of those individual file records.[/quote] Right - but that huge advantage was lost when Apple decided to drop AFP support and didn't add such a feature to SMB. (And this is part of my current pain having to find alternative search methods (https://findanyfile.app/fs/) for various servers, such as Spotlight, SSH login with "find" execution on the server, using Everything's http server on Windows, and so on. It was all so easy when we had AFP). Is there even any supported network file system left that supports searchfs? Does NFS (I believe I never checked because it's so diffucult to use)?

See also: https://github.com/KhaosT/FSKitSample and https://developer.apple.com/forums/thread/765424

Wow, Kevin. That's excellent support, thank you so much for making the effort! I had no idea that using concurrency in file system calls would have any such effect. You proved me wrong. In a real implementation, I think you'd probably want a fixed set of threads calling "getattrlistbulk", another set of threads processing the data each call produced, and then system the pushed directory back to the syscall thread. Already doing all the processing past the initial gathering of the directory contents in a separate thread, but that's clearly far from what's possible. I actually tracked this bug down and I have a question about your testing. Were you always testing on "freshly" mounted volumes (meaning, umount volume-> mount volume-> searchfs)? Or did you also test on volumes under normal use and repeatedly running searchfs? The use case of my app is to mainly run searches on the entire startup volume. So that's what I focused on with my tests. I try to be aware of caching effects (I believe some OS versions can use a of the available memory for caching information, but never learned about the details - all info on this I found was fairly vague). What I did, for instance, was to run searchfs on a running High Sierra system using HFS+, then cloned the system and let it convert to APFS, booted from that and ran the same tests. Repeatedly, of course. Later ran the tests on upgraded macOS systems, showing no significant improvement. The thing that confuses me most about searchfs performance on APFS is that when I search for something that's not findable (e.g. a unique name) both with searchfs and with a conventional method that eventually goes thru getatrrlistbulk, it didn't make much of a difference - even though the number of syscalls should be immensely different (and thereby contributing a lot to the overall time): searchfs should be a single syscalls, whereas the other method will be in the 100000s, right? That is why it could be so fast on HFS+, because all the searching happened right in the VFS, in a tight loop. Am I misunderstanding how it's working on APFS? I always assumed it was because of the dual-tree nature of APFS, where one cannot simply look at the one tree containing the dir records, looking for matching names, but must also look up the corresponding entries in the other tree in order to figure out which volume the dir entries belong to, and this would be the bottleneck (though one could argue that simply first blindly looking for matches in the dir rec tree and only then verifying the other tree if a match is found may be faster than the other way around that I suspect it's happening now). You understand what I mean? searchfs may be optimized to either first sort out the right dir recs by making sure they belong to the target volume before checking the dir properties, or vice versa, and maybe the other way around would be usually faster (assuming one seeks to find a small set of hits). Of course, that would require more work on this code for very few gain (hardly anyone is using this anyway, I believe). FYI, "find" is opensource and seems to be a relatively straightforward implementation built on fts. Yes, I know, and still my fts code in DirScanner was slower. So I assumed it had to do with the use of NSURLs. However, I must now admit that I was wrong in that. While find is still twice as fast compared to the search performance in FindAnyFile, it is now on-par with my most recent DirScanner version: After Jim Luther told me about the key to using fileSystemRepresentation instead of converting paths to NSStrings in my code, DirScanner became significantly faster. Yet, I failed to contemplate that fact when I wrote above that fts would still be 2x faster, or was too lazy (or confident) to double check. Now, with your findings on running parallel dir reads, I have a big task ahead of me rewriting that code. I'm excited. Thank you again for all your input.

Kevin, I appreciate your involvement in this, and your willingness to dive deeper into this for my (and hopefully others') education Some of the things you explain about performance were already clear to me (I used to write file systems and a simple operating system in the long past). Some were not clear, e.g. I have assumed that maybe some of the NS based calls were happening in a kernel space where making calls to the lower APIs would be less wasteful, or that NSURLs, which these calls return would be pooled at a more efficient level (where I only recently learned from Jim L that it was for me using NSString for paths where the fileSystemRep was the key to keeping it faster). I also agree that some of the properties, such as "NSURLIsVolumeKey" shouldn't be costly because they should not require another call into the BSD/POSIX APIs as I'd have thought that the information is already present at the level above the VFS. And that's especially true for the NSURLIsDirectoryKey. Architecturally, the performance here should basically be "identical", as the ONLY reason "hasDirectoryPath" can work at all is that the enumeratorAtURL "told" the URL it was a directory at the point it was created Exactly my thought - I concluded that from logic, without looking at the source code. You're aware that NSURLs append a "/" suffix to dirs, while paths do not, right? So, I suspect that hasDirectoryPath simply checks for that trailing "/", hence it being so fast. BTW: If you time the "find" tool, you'll see that it can be almost twice as fast as my DirScanner tests. Which surprised me. I still wonder if NSStrings conversions play a role in that, which the find tool doesn't have to worry about. [quote='830058022, DTS Engineer, /thread/776198?answerId=830058022#830058022']the only explanation I have is that there's a serious issue in our current implementation and the resource cache simply isn't working properly. Please file a bug on this and send me the bug number once it's filed.[/quote] Huh, well, okay. I was already expecting that anyway, but never bothered to report it as a problem because I didn't think anyone would bother with it. I had reported, for instance, an issue with searchfs(), which has become much much slower in APFS over HFS+ (like, 6 times slower for the same amount of files, when it's an entire startup volume), and this was not addressed, as far as I can see. And if we're talking about performance issues on the FS, the worst offender is Finder and its support frameworks. I have once run a trace of calls (e.g. with fs_usage) and I found that there's a LOT of repeat calls for the same attributes, which explains why nowadays browsing the Applications folder, especially over a network volume, is so awfully slow (it's mainly about diving into the bundles). THAT needs addressing, and it's so obviously bad - it should be obvious to anyone using a Mac for a few hours, especially if they have some experience how fast it should be (and indeed used to be!). As long as that isn't tackled, I wonder why I should make an effort to report these rather minor bottlenecks. But then again, maybe, if enumeratorAtURL is used by Finder a lot (can't dtrace anymore, can we?) and and this gets improved, maybe it'll automagically also improve browsing in Finder all over, significantly? If you think so, I'm happy to make the effort. (Feel free to email me directly if you have private comments, I don't want to start a flame war here, but rather really care about this getting better, if I can make a difference. Incidentally, I had applied for a job a few times working on FS improvement in the past but nothing come of it.) [quote='830058022, DTS Engineer, /thread/776198?answerId=830058022#830058022'] my advice would probably be to build and optimize entirely on getattrlistbulk. [/quote] Yes, that's indeed on my wish list, after seeing how the find tool can be so much faster (my app has now a mode where it calls the find tool and then my app just looks up the paths it reports back, and that's twice as fast). But then, I also want to improve the UI a lot and add a caching of the FS by tracking all changes and keeping a shadow directory, and that'll benefit my users more in the long run). Only so much a single programmer can accomplish :) using multiple threads and making multiple calls into getattrlistbulk simultaneously Yes, that's on my list as well, and yes, especially for network vols. Also, I had already planned this before SSDs became common, and I'd thought to identify which drives where on the same HDD, and only run concurrent searches on separate HDDs, to avoid excessive seeking. Though, I still have to experiment with this, because I'm not sure if the VFS would queue calls on a global level - but in recent years I got hints that this would not be the case, and I'd be able to have multiple FS calls run concurrently if they're on independent file systems, and you seem to indicate the same. Which is promising. However, most users of my app are simple "home" users who only search on their startup disk, and there's little to parallellize there. I had, for instance, considered to pre-cache the locked system volume, but then, the time for searching that volume is fairly quick compared to the /System/Volumes/Data volume, so caching the former doesn't gain much in the overall search time once the user has accumulated lots of files (which happens eventually). Again, thank you for not only trying to answer my questions but also exploring solutions.

Kevin, I'm using both a simple recursive iteration (contentsOfDirectoryAtURL) and enumeratorAtURL. In both cases, before I read the dir contents, I check if it's a volume with a trigger flag - if it is, I call opendir, readdir, closedir before I invoke one of the above FSFileManager methods again. That way, at least in my use case, the sub vols get mounted right away by the opendir/readdir, and the next filemanager call provides me with the new volume's contents. I only have one user out of 10000s who has reported this issue so far, for the >15 years I have my app in the public, so I consider this a "quick'n dirty" fix that works well enough for now. I've noted down your comments, though, in case I have to work on this again. I have a concern about performance, though: I like to be as fast as possible with the iteration, especially if I only need to look at file names. So, when calling enumeratorAtURL, I fetch no extra URL properties, and detect directories by called url.hasDirectoryPath, which turns out to be faster than checking NSURLIsDirectoryKe. If it's a dir, then I call getattrlist to get the mountpoint values. (See my "DirScanner" test project at https://files.tempel.org/Various/DirScanner.zip, which also employs this technique to avoid diving into other volumes, e.g. when browsing from "/" down.) Any idea if there's a better way? (I also learned recently from Jim Luther that it's much faster to use the C-string "url.fileSystemRepresentation" path instead of first getting url.path and then getting its UTF8String because that skips come costly internal conversions. Same for creating URLs when the name/path comes from POSIX/BSD functions. I've updated my DirScanner code accordingly.)

Ironically, the answer is found in a 10 year old post by myself (https://stackoverflow.com/a/28663838/43615). Basically: Before diving into the dir, get the ATTR_DIR_MOUNTSTATUS attribute and check the DIR_MNTSTATUS_TRIGGER flag. If it's set, it means the dir is currently not mounted but will trigger a mount once it gets read. So, if the flag is set, I only have to read the dir's contents (which will report back as being empty), and then I can immediately read again, and this time the mount will be there. I thought that there would be a delay, but apparently this happens synchronously. Alternatively, if I do not want to check the flag beforehand, I could also, if I find an empty dir, check if the dir is a mountpoint (DIR_MNTSTATUS_MNTPOINT flag) and if it is, re-read the dir.

I am in the same boat and am now confused - I thought the solution, for ObjC, was to use SKReceiptRefreshRequest to refresh the receipt?! My personal problem, though, is that I cannot test my code now. I seem to need a fresh system that has never seen the app before. I tried to create a VM with Fusion 11.5.3 as well as 13.5.2, but in all cases, I was only able to install macOS 10.14 or 11, but then upgrading to macOS 13 or 14 always results in kernel panics during install or shortly thereafter, with all settings still being defaults (other than increasing CPU cores and memory). Of course, this needs to be done on an Intel CPU Mac, because the Silicon Macs don't support the App Store in VMs. This is so fooked up.

Done: FB15568200

Aaand I've confirmed it: After restoring the VM to 15.1beta where FAF hasn't been launched under yet, then deleting all copies of FAF (which are easy to locate with my "Launch Services" app) and then installing the single new version that contains the new key, I finally see the new usage description. So my suspicion was probably correct: When prompting the user about the Local Network permission, it doesn't fetch the NSAllowsLocalNetworking value from the running app but from Launch Services, and in a way that doesn't ensure it's looking for the data from the latest or running version of the app, which it should. Which isn't really a new issue when I think about it - I believe I've noticed more than once in recent years where Finder or other services failed to register new information from the latest version of an app but instead use data from outdated versions that are still installed elsewhere. I wonder if this is even worth a bug report. It might be behaving "as designed".

Nevermind - I found a test project myself (my own SSDP Browser on github). There, indeed, setting the NSLocalNetworkUsageDescription does work. After verifying that, I checked my main app FindAnyFile again. Lo and behold! After changing the app's bundle ID, it suddenly worked! Now, the issue here seems to be: The app was already known to macOS BEFORE upgrading to Sequoia. When running it on the updated Sequoia for the first time, the prompt for Local Network appears, but it's showing the default and not my custom usage message. However, if I install and run the app for the first time on Sequoia (which I simulate by giving it a new bundle ID), it does show the custom message. So, I still believe that there's a bug in macOS 15.1, by which is doesn't show the app's NSLocalNetworkUsageDescription if it has been installed before, or possibly if there are also OLDER versions of the app around that do not contain the NSLocalNetworkUsageDescription, meaning that this information is cached somewhere else by macOS and is not updated with the LATEST app's information. Can you tell me where macOS gets the NSLocalNetworkUsageDescription text from? Apparently, it's not reading it directly from the running app's Info.plist but from somewhere else, e.g. the LS database? If it's that, I could dump that to check why it doesn't get updated - or maybe it's updated there but the macOS code looks for the first app instance in the LS database instead of looking for the latest. Something is clearly wrong there and I'd like to help figuring out what, so that I can provide a clear and reproducible bug report.

Actually, the cause might be that, since I'm using a NSSearchToolbarItem on macOS 11 and later, I need to invoke its beginSearchInteraction method instead of making its searchItem the first responder. While doing the latter used to work, this has apparently become unreliable in macOS 15. It's weird that it sometimes still works, though, making this tricky to notice and debug.

Thank you for double checking that it should work on macOS. yes, I showed the plist of the app that I had launched. Not localizing the text. App is signed with dev id and notarized. Do I need special entitlements for this to work? App is not sandboxed if that matters. Can you share the app that you verified to work so that I can compare?